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;;*vqcabu^^^ eh'joyraent^, anid plot cpm'prehension ..f or ; 
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'Series', -A was evaluatf^d in-' four, studies/to determine: t 

"ef fects.- o viewing without teacher . iiiput,' wit h teacher^ input, 

ivbet^w eeh; q^^ and attitudes toward thW 'program* 

■ Evalu^^^^ language, materials are 'def ined and 

^fOisted-^i categories: those factoirsvwblch. can be exairrin^d 

yindependentl^^^^^^^ requiring viewing the - program, and^ those which *^ 

need;:.viewer''interaction with the' pfbgijain- :a list' of references is-. 
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ABSTRACT 



-rhe paper presents the resxilts of tvra .f orfaative> evaluatibn studxes, 

■ ' ■■ ^-r r[ ^ ■■ V' 

:'. of second language prograihsv in Nova Scotia. Both studies i'riclude'd ^^^> . ^' 
■ procedvires to determine the effectiveness of the series in adiieving. ttiexr 

. ;> objectives as well: a'^^ procedures which cduld yield suggestions for imrr^ p:-. 

•/ '^V^', ' - * • • ■ ' . , . . • . V 

'.proven:ent .xn- the. programs. ■. '.- - ■ . 

- V Ji! second 'as|).ect of the paper concerns .-^e iirp^rovement of formative ^^"^^^^ 

evaluation- procedures ^pr television and. other mediated programs. The 

suggested changes are divided into factors; whictil?. can fae exaufid^ /- ^^ 

; independently/ factors which must be examined in context and: factors r :^^^^^^ v.- 



which musi be examined in a Ifield tryout. 



■ . .J 

1 ^'j 



-I 



0 :y 




■■■■■\ 



A f educatj.6hal. media <ha^^ a. definite^role .to play in helping ^children 

and auults'-to leaim other languages. Radio, audio^-tap^s, films and slide-, 
sound packages and television tiave all been used to achieve vn:!^]:Ov^ i<.arning -"'-i,;'; 

• ' - -^^^ ^ ] ' ' . - 

..objecjtives. Many ^ of the- ear ^y language media presentaHons VciJ?*vi=^-:*KnJ, of " • 7*1 
r . stand up teaching .using blackboards and perhaps a few visuals"-b^A^ \f'' U:?' - • i 
, else.- However, with tihe maturation of educational television. more'.vaivJtls- 
ticated presentations are being .designed to teach languages. - - " -! 

: , ^. .. Many: agencies have' produce<Jv:;eries for - use, in second v language teaching:^ q^^"^::^ 
: The; .Ontario Educational Communications Authority , in -Toronto, has pro<aitG^^ 
: several series for. use in teaching French' while; the ^CarrascolWidas seri 



„has. been used to teach Spanish in the United -States. Nova- Scotia School 
Television also produced several; series to supplement : teaching of- French 
■as a. second language. . v . . . . v 

Most of the recent television series intended for se^cond language 
learners have been subjected to fprmative evalxaation orV at i^st^sunroative^^^^^^ 
evaluation procedures.. These studiesfhave dealt"^ with various -aspects of } v V 
the production and utilization of secondN language television material's. - - ^ ^ ' 
(See Laosa, 1975; Williams,^ Va^i Wart'' and Stanford/ 1973; ^ Fleming, 1972) 

. ' ■ ' ! V-^~ ^ ■ 

_^ This paper deals with the improvement of fprmative evaluation pr^ocedures ■■ 



. for; second language television - materials . The recommended procedures are • ^ • *^ 
; ^ based on a review, of the literature and on two^ studies of second languag^' ^^v:;' ;^ ^^^^^ 
. • teaching materials produced by Nova Scotia School Television ^-(Lewis and - 

Fisher, 19^77; and Lewis, 1979) . Although the paper deals with procedure^ "\ 

fqr television programs/ the comments could be applied to other ^media and • V ' \t 

oths:r 'programs. ' r . ■ (jT""' -iW'^* 



'^:fe> -Ohe Nova : . 



The vrecoinmehdations on^ improving procedures 'are based on two s t.udies of* 



•I 



?^ ; second "language television programs . To provide the ^reader with this 



,V::- >^ :b^ siammari zed below. : " 



'^■■.--:.v .:Vive Larr^Compagnie- : ^ • ■■ ^ • ' ^ • 

^i; •■ -. . ■ - •• ■■ . ■ 

. '^■Vive La Compagnie' was a -second language French series^ designed for • • 

• •Oj: • . - . *; • - / ■ 

: • . gra^ students.; - In each program, • three actors (and other- supporting . 

^■^^^^^ in real-life e5q>eriences which might- be faced' by ;. • 

..rip ^ vocabulary and relied on humor _ 

* ' . ;' cLnd visuals to convey meaning. - * ■ . . * > 

• ■. ■ ■ " ■ 

The ,.Evaluatxon ■ • .. ^ ^ \. ' 'v'"' - 

i: ; The /evaluation was commissioned by the Nova Scotia Department, of ' 

^ ' Education to the eff ectxveness of the program saries achieving 

■>- ■ : ;■ 7 its'-j^^ .which dealt with; vocabulary "acquisition^ enjoyment and 

understanding of the plots. .In addition to determining eff ectxveness , the 
Study had to provxde specific information on recommended production chctngies; 
distribution patterns, usage guidelines and supplementary materials. : ■ / 



Specific Ques1:ions 



• ' 1.. ' Would the students' watch» rhe programs^ i.e.* would their eyes be on 'the screen'r' : ' p^i 
. 2. ' Would the students understand the words, to be used in each prograim? , .^r^^,,:-^^ 
.{(3. After T^atchxng. the programs, would/students understand words which were 



.. ., , . . - ... „. ... , ... . ., ^ ,, 



• , not in' tlieir repertoire before 'the viewing?: ' / - " ^--o ...^ • 



. /J- '. 4. ' Would students understand the plots? 
^ 5 ■ students ^n^oy the . programs? . 



"-iTTl -jv r^-- V - _ _ — ,^ _ _ x^.'i_/. , "I ^ , . ' (^^■■\'^\J'!!\f^ 



: 6;.', . ; •: Would .• the;| niimber : of progranrs ; watched result in differences in; v6cabula27; 

•1;. 7.-7; . . Howrwoiild" students reacts to seiected! variables - such as . the^..plot, . ■ 

. production "techniques? , ' ■ - • * - . 

; :. 8.. ..- -.^ Would.-,^^ learning -French and towards French • ... : 

'^^^^^ ^-^"^^^^^ ' * .P to the program? •/ 

; ■ 9. ^ would the' teachers feel about . the prograin^wit3i regard to its ..^ ^^ 'V 

''u- '■■-■^} level,, its achieyeitieat of objectives? etd'. . ! . ^ - , ■ ■ . 

y ; batav Sotirc es . . , , ^. ■^i;,..;* , . • 

y. J:^* . used three basic sources of data: ; expert opinion, 

; -student 'opinion ^ results, and teacher opinion. The expert opinion con~^ 

■sis ted of . a. contextual evaluation in which the programs were scrutinized to . ' 
de£erinine the production [variables which could be improved. ' The, act:ions'"6f v 
, the characters, the pfonu^ special effects all came under 



close scrutiny for suitability. ■ . in addit the ■ 

-production format, etc.; to. existing research, recommendations for children's - 
'^v - V'P ^investigated.' TJie^ researcher ^determi^ied- wheth^ the '-^ > 

" recommen^^ in children' s programming were, being considered >; ' 

■■ ' 'in' , the. Series." ■ - ■ ' " ' ■) ■ ^" ■ ' ' ' . . ; ■ " i ? ' ,v. I 



Teacher attitude was' obtained using, a Likert . type scale similar-^ to the-^ .v 



■ students'/ scale, In addition*, each participating teajCher was -interviewed 



to 

determine*-: his/her reaction ( to, every aspect of the program which could, if 
n^cessaar/, be:^ changed or altered. 




Xlasses were chosen at random from all the grade -eight French. classes in one,. 
Nova; Scotia . county.; To determine the ef f sc;ts of watching more programs;- : " 
some r classes watched eight programs>' some watched four and- some .watched one " 
: program; other classes did not watch* any programs but parti ci5)ated in the 
■ study so that valid comparisons could be mad^. All classes completed a: :V i' - 

baseline test battery before the study Began while one-half of . the sample . ; 
:^ completed ; a pre-test. Classes viewing the program watched, it on videotape . v 
^ ^.in - their classrooms di^ the ^gular^- French class*" Discussiom of a program 
%follQwed its 'showing. After* watching the. appropriate number of programs, all 



the classes ;wer^ tested on *the same program using the techniques mentioned 

' ' ' - ' 

above. To obtain the most representative sample of teacher- opinion^^^^^-^ - - 
Regarding the program, any teacher using the program outside the study as — 
well' as several other groups of teachers completed the teacher questionnaire 

,/-on.':-tKe '.seaAes.- ■ ■■ . .■■ ^■v. ■ \- :/<''j.''^ ■ ■ 

■ ■ -.1-. ■. ■ ■-' f^- ' ■ - . ' ■ ' ^ . ■■ : 

. -■. ■,.*...•.■- :. ■ ■ ■ .. . ■ • . . ■ i-:., - • 

■ ■ ■ . ■ ' ■ ■ ■ ■- ^ .. • , I ■■ ■ ■ : . ' ■ ■ •• 

Pre-Treatment. Tests for Differences ~. ... - 

^^^^ had to be randomized, not students. . As a 

result,' differences- between classes -had to be measured before the treatment:^ - 
was presented. Two .tests were administered to- the sample prior- to the i study.* 

. ' The Ici on Parle Test, an audio-pictorial Tvocabulary test, discovered • 

• I ■ ■ ^ _J „ • •/ . 

* significant differences between groups. In every case, means for o the TV ; 

■ ■ ■• ■ ■•. ■ . ■« . ■ . I .. . ' ■ .■ ■" ... 

■group appeared to be higiierr'-:than means for the non-TV ^ group. ' - - - v ' 
. -Significant differences between groups were also found in responses v. 
to the questionnaire on attitude to learning French. Th:ls test comprised two 
s\±)-tests, Desire to Learn French and -Motivational. Intensity. ^ 



ResTilts* 
Attention 



•j-^ The -programs maintained the attention of students .watching about : ' ^M^^^^ 



; 80% of the time* 'Hopever^ certain sections, of programs did seem to 
^.encourage students -to stop watching programs, in addition,^; it appeared ; 
that students who Sid not -understand the action (determiijned^by ^self -report) . ; 

■ . ■ • •• . 

_ :^topped watcKing to catch up on their^ sleep,; homework, etc. Sections ;; /' 

of programs wh^-ch* were accompanied by lapses in attentioii were noted for . ^ 
••. :..further/-analysxs.. . . ■ . :. ■ ■ > 



-^ Vocabulary Level Before Viewing 

0^ students who ^ completed the pre-test knew 14 of the 20 words . 

.on the test. This data indicated that studejits could understand the major 
words and sentence expression used in the, program. This finding • showed . 
that the' assessments by the* teachers and of the producers regarding vpcabulary 

; level were -reasonably accurate. - • • 

Vocabulary Level After the. Program > . ; 

r '^^-: '- -.T^^ between pre-test and post-test vocabulary ^scores t 

indicated that students 'did\not learn new .words or phrases, as a^esult 
^ I r / 

, of the prog<ram. Neither the, pre-test group/ nor the non pre-test group , - 

who watched the television /showed a significemt difference on the post-test- 

from the non-television groups. However, it should be pointed out that :r 

the program was not designed to teach 'vocabulary. :^ 



*.; This section contain summary information since the paper is concerned , : ^y,> 

mainly -with tlie ii^ Complete results are pr esented ^ v^^v^J^ 

in Lewis and Fisher (1977) . ^ . " , i 



J . ; . , , _aaie, datk>md indicate that a-' student who had- been exposed to a preP-test 
TO higher than -a student who was riot exposed to • . 7- , - . 

^ • • .. ' 

-^r-the pre-^est. : As a resiilt,. one of the recommendations suggested. that as 
• "^am.instruci^ students be* exposed to the vocabulary of the. • ■* 

-^ prograigfi before it was viewed. . . 1 

Coinp"rehension'~df~ Plots 7 - — ^ — * 

/< A/^^udents who watched more programs understood^ more of the action than 
."students who watched f ewe - It is obvious that comprehension 

^improv<id as the students became-^abcustomed. td; the^^accent and pronunciation 
of the characters in the Series; These - findings clearly demonstrated that v 

. r •. ^ •; ■ . ^ ■. 

viewing : the whole series would be more beneficial than the viewing of i.---^-: 
, just one* program, or even just a part of the .series • ^■'h-: 



Opinion of the Program • ■■■■ ^- ^ - " - ■ » " T 

^ ; " "No significant dif ferences were- recorded between students who . watched 
.one, . four, or eight programs. In ranking' the programs /»v'students chose ^ 
programs which they said were easy to understand, funny, presented • ' . . 
pictures *to explain the words and had a gool plot. More information- on . ; 
these factors was sought and obtained in. .the group and personal interviews 

, •. .. ■. ; ■ ■■' . ■ :^ . ■ • ■ ■ ., .■ ■...■.„ ,- !■ . - .... ;. ' ■. .; V'■:■■- 

, after each viewing and at the end of the study. The findings of the 

interviews and the siommairy of the questionnaires provided the basis for : 

4icuiy of ^^^prpduction changes which were suggested. ' '^^^^^^ ^ ^^^^ ^^^^^^ ^^^^^ : \ 



-OpinionS'-'of .^.TeacherS"' • • : , 

/ . i - : : M would like the program but , 

/3that a considerable amount of preparatory rand follow-up work would be 
; -rieeded;.^^-' to felt that the students would not understand -the 



- ?i 



. 'W-j- ■in.' 

,.-.. .1,^ J. 



v. vbcab'i^ was 'completely contradiclied' by the '^ '^^^^^ 

^^5^ students".™' A strong. • feeling- of the " teachers was- - • :. 

;-.*'thatr:ti^^^ of one of' thie^haracters "would, inhibit iearhiiig the 



: . correct; proniihciation^ V 7»s a result> a change in characters -or an •iiii)rovement^^^^-.^^ .^^^ 
>^n; pronunciation, were suggested* recomitvendatiens . ' ' ■ ' 

CAtti&de".;.'''' ■•:^■'■V^*•^'3 

t^K;.-:^ :<The:-;vi€iwing of the- prqgram^ma.de no^ dififer^ce inr the' attitude :scores Yy^-i''- f^r.y^$ 
: • of students. Those who. watched :dlcl^ no attain higher attitude iscores.' " • f; ' -'^1 

■;■ ■ . . »■ '■ . * ■ J' • ■- ' ■• ■ • 'l^.r ' . ■■■•»■:. ..... ^ _ , » , , . ■ . ,. . , ■ , ■ . . ."^ ■ ■ — • ". • * . -.'i- . 

J Probxeins {With the Study ■ ' - . : - : , : >; ■ '^^^^^^^^ 

Experimenter Effect ■■ ^, ■ . \^ •;..vV;=i:-4*i 

jr 'f^^ In this- st^dy, the experimenter; supervised the viewing of: all programs - ^v'^^ri^-i^j 
; : arid adminis teredo all .tests . This procedure could, hardly - be called a • ,:■ ^ ' V i '^i 

;: natural situatiJn for the .qlasses involved. , As a 'result, the generalizabili'tyj/.- J^^^ 
- of the .dctta to a teacher admir;lstered situation could ^ be called into- queistiori. 'ki^^^^^ 



Vocabulary ' Testing 



' T vocabulary^ test was used as the pre- affd the ppst-test./ ; 

^ The data ..indi^ that the, adminisjzration (of the pre-^est 'affected thev":^' 
•V .'post-test scpres> i.e. /'students who -took the pre-- test socred^^ 
/ higher than -^students who did- not complete the pre-test. . v; 



Attitiade ... 



' .A ttitU'de,! measiire. ' J;owever/ with\the efforts of 'f . 

; ; ; Gar dner ; and Smythe • (1975) ' attitude towards ..:learjiing French .can J:'be measured 
llHowever / barSne Ginsberg and ] Smythe (1976) .noted that attitudes :"of- ^ -^'i |V> 




' con taxned . a ..number of . short segments which pffere^- a range^of topics.; • The 
• range of segments; included appearances *h)y a teenaged host and hostess, , . 
singers, a ^ band, game shows, activity programs ; and on-^ite visits to ' 

.A'.,;:,-. :;-v:-^; ■ ■ • ; - . 

; ; locations /jlike Cheticamp, a .fxshxng" village in Cape Breton. . V « , ' 

> Supplementary Materials a 

; . Supplementary materials included a teacher' s guide and a set of . . - 
'. .flashcards depicting key vocabulary and concepts used in -the program^. 

■The! vteacherVs guide ^^i^ list of the program' s objectives, 

; ; virtf orination on the segments, presented in each program, a description of ^the 



^\?Hf'^ -i^^^^ vocahulairy and idiomatic expression. > . 

^7 guide, j however, d^ any suggestions or directions for vising .. 

■ ^:.tie::program.--;>::- : 

,; . .V^ depicting vocabulary and 'Concepts" were available 

:-free,of; charge to teachers requesting them. These flashcards were printed 
. in. -black and white on heavy cardboard. ^ - 



: Field/Testing-by Department Personnel ' * ^ . *i • 

: : 'Three - of the .programs were viewed by students in' 20 schools during - 

■■■ I: .y:^ . :" 1 :• V*'*'.'-''- • • .'v ' . .•■ v-.^V-:',- ■ ^ ■ - • • ... 

visits by a staff member- from the' Education Resource Services.' After the 

.students viewed the program, they were asked to comment on the programs dn 



an open ended questionnaire. Some of these comments aire presente*d~in-*a-paper— 



by - Johnston-Doyle and Lewis (1978) . 



.. Purpose of the Evaluation ' ^ ^.-j — 

te^^The purpose^ o the. evaluation was to determine .the effectiveness of- 
' .the series, in achieving its objectives and to make' conoretfe recommendations 
for changeiss^in the- production and utilizations* of the: programs. . 



I 
I' 



Specific Questionsx ^ - 

' ; 1.^^^^^ ^.W^ have an effect. on students' ability to 



...v. comprehend wrdis and sentences? 



: 2. ; . students recognize correct; constructions? 
, 3. ^ Would the Students understand, the content presented in the segments? . ' 

fli Would- the students identify with the teen--aged host and hostess in the- 
•v. ^prqgrauu? t ^ &^ . 

'■ 5;- ' iWbuldV: the students like the musical segments presented in ;the program? . 



French be affefcted' by Viewijig ^"^^ 

by ^'Viewing'.-the ^progfiaih?.^'" 
■"i^^'-^^^^^^^^^^^ :teacher ' s ; preparatory^;and •fbllqw-up 

■^^;^"V;* 5^ recognise, constructions.,^ 

M^' U^^^^^^^^ ^9l\ leamihrj'- Erench and • attitude tp^^e^^ 'class? . ■ • ' ■ ■ . V ' ; : : 

program could be recommended. a^^aJr^sxilt-^-cif^the-^^ 
=: ; vi:;^ 0.i^':i:-^siajng Kand- •*j:lie /opinion of teacher and students; *• ' -aI ■ - • "r^ 



vj^^r;!:^::,^/ :v Siyitied into four sub-studies ''^ order to ;answer.,the'' • ^^ -i 

! CV by the clients., The first study^ .concerned ^i^seif ^ 'J^' 

y^:Z':0'-:r-:;^^ viewing the prograr^s wilJiout^^a^ input. The 

-^^^ monitored, the ef fects :of ;the. television'program '£mdx teacher ; 

^ ^ "^I^^^U?^^^^ ^tucty dealt with the; difference between the two groups^ : v ' lii f! 



: s : : in the first two studies The findl s^tudy surveyed ' /attitudes .towards J; the 

^' ■ ; !■ i... , . _ ^- * . ^ .. ■ 'f^ .... :. . ■ ■ • ..• 

' ^-^ ■ • . : ' ■ . ■ ' ' ' '' • ■' ■ • 

rv^v ':..-:':'' :Sainple^ ; - ' / ■* '■• •■: •* 

: y used populations df. grade eight students inHNbya .Scotia.^ - 

^yy:Z'-t.'[/^- In studies ione and two, /an experimentalr-gi^^ group (Viewing/non-r r 

: - r ^- viewing) 'design was usealwith^t^ 

ij,-. V group and a-'non-viewihg/^grp . 

(78 students) watched ;the:'^prbgrajn.^^^^ the control^ 

. group (92 students) continued with'^thel^ curriculum* -In study . two, 

V • ■ the 52; viewing students watched the program and participated in teacher, .-r^^^^ 
*. - ; prepca:ato2Y:.and fbllow-up activii^ while the* 20.jcontrol group- students 
-'^ •■ continued rwith the curriculvimi \.,In study three, the^ 52 . sjtudents whp' viewed 



3^ 



ERICl 



|v\ff.S-^^.b^ the-^78" students 'who did not have teacher ■ • ■; 

V'^^^^ sought to measure.: attitudes and opinlonsV included"'; 

=py';:;^.;:^>-''^:;:^^ viewed- the: programsv. ■ " ■ T ' ■•■ ■ ■] 

■■■■--A:,' ■'■ .Procedure •• ■ ■ . ■"• . ' ^ ■ " • ^ :"i-\-:-^:k^<^ 

' v\:>S- battery which included a test of French 

^^rrr^T~4^ a basic info raat ion ■ 

^- ^ The;ir. viewing ^groups' watched eight, programs while " non?- viewing . 

V;' ; - . . : groups proceeded w^ work. After, viewing the programsv all 

y ; ■ 's completed the post- test battery . / Vi'ewi^g -students, com^ 7 

: " - a test,- a constructions test, a slTitence comprehension test^ > • I' 

. . ../'^^ tests cind^the tests^n the ;opinion . 

; .o the program. Non-viewing , students completed 'only the 

r'^y-^;.^... ..;y- ■ ' .*^,^. ; ' •■■ >■.. : ■ ' :y . \- \ r . -i.- 

-vbccJ^ulary, sentence comprehension, constructions and three attitude "tests. ' 

v.y' .-:y---. * >■•■ ■ .■ '• ' . / _ , ■ .., • . V 

- " i .1- .. ' " ■ • ■ ' ■ •. ' ' . - " • 

■■ ■ . . • , ^ _ . ■ ,■ ^ • ^ , • 

. ■■ ■ • . Analysis : . -. . ' ' ■ ■ * ^ ■-. •i-'^ 

■ ' ■ ^ oiie' analysis o.f covarian developed -by the Division- of ^ • 

/\' : ; e^^^ University i3f Alberta ' 

.■ . : ■ • ■ ■ .■ " ., .. ■. ' ;, ■ ■■ ■ ' Z- * ^' ' ' ■ • 

j r. ; . ' " The analysis is computed' on the basis of .a pooled 

-regression equation, homogeneity of regression is assumed.- This procedure . \ 
■ L V ' ; V was used' for stu 3. • ' ' - ■ . ■ . . 



0 ': 
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Results , . ' ' . ' . , ' 

Study r • ' • ' I, ■ : /' ' 

vThe pui5)pse of study 1 was to determine whether exposure to _the Series .' 
woHjldv.result vin- an increase/ in post-test scores on ability! and attitude. : . 

:••■;* ' '^v-:. "a-- •••• ^ ' ^ /V « ^ ^' . • 



.... ^ * '.\ . / 
.. Baseline Measxifes' . "X *; ^ - ' ' " . 



■ 'jj- •. ■■•/ ■ ■ ■-v■ 

c: >" The i two groups (hon-yiewing and viewing) were not sij^nif icantly 

• different: on the baseline measures.^" (See table 1) . On the ici on 



/ 



-:,vi-. 



Parle, Test, a test of French •'ability, .'the viewing" group : scored 25.31 



./ 



Vhile the control group scored 23. '65. . ' Both groups scored betwefca 60; • • . 
and 70. percent on the test. On the attittade test the viewing group scored 

•• 60.24 while the non- viewing 'group scored 56.53. : . 

. " ^*// ' • . . ' ' - • ^ ' . 

, :','/' ^: ; _ , ; ' • ' • ' • • ; 

^- .Post-test ■■ Measures' ■ ■ , , — \ • 

The scores* of the ^two groups were significantly different' only on the • , 

. . y . . . ' r\ . . . ■ ' • ' ^ . ^ / - ^\ 

ocaJDulairy . test, after adjustment, for the^ effect of the covariates 
measured in the baseline test. ' (See tcible 1) i .The groups were not ' 
significantly different on the. sentence comprehension -test', 'the construction 
test and the "three attitude measures . ° v ' > 



'Table. 1/ i 



Unadjusted,' adjusted means. and'tF. Ratios 'l ■ 
■for the '.viewing: and ■non-viewing,gi:oups: on all 'tests 



'Test: 



•Ici on Parle Test " 



(Max=40) 



0. ■ , 



J 



Unadjusted Means 



Adjusted Means 



Viewing 



.25.31 



Non-Viewing. ' F. Ratio df , .Viewing/ Non-Viewing T..- Ratio • df\ 

• ,■ .. „„ ■■ „ - ■ , I— p ^- . . ,. , •- ■ .,. , . . ,A/..:vfe! 



■23.,65 



2.62: 1/168 



;;.J';./'::Attitude:Testl '. i 
(Max=90):' ''- .' j ■ 



60.24 •■ 56.53 ; '3.07 ■ .1/168 



Sentence CoKiprehension- Test 

'(i|x=ii) ; ' 

Vocabulary. Test ';'V 



4.74 4.18 



5.97 



.\4.43 

A- 



3.84* .1/168 ■ '4.57 ' 4;;33 



8.26* 1/168 ' 5.59 ., '4.'i6 



5.20*. 1/166 



Constructions' Test 
(Max=14) ' 



6.06' 6.20 



.15 1/168 . 5.! 



6.38 ■ 2.80' 1/166 



'^V /r , Attitude Test' 2 
; / * (Max=61) . • - 

Attitude Test 3' 
;'.• ^. (Max=155) ' 

"••'Attitude Test 4 ♦ 



'» p>.05 



'...1. 



42-.49-. ■ ■ ■ 38.75 



90.17 •■ 87.,18 



104.18 . . 9,7.46: 



6.87* 3r/168 '"41.66 ■ 39.44 



.0 



3,21'' 1/166 V:' 



1.37 '1/168 ^90.46- ".86.9^^^,i' 1.85 v' 1)166'. 



4.05* ..1/168. 104.09 97.53; ',; • 3.7V-''':i/166;:,;,.;:'>- 



1 - '.I. 



.|. . . 

. '2,1 



•^r .:;':\-;';i,.'.l'-';;./.:Vi:hyf!S 



* ^ ^ • The' purp was' to determine whether, exposure ta^^the Series' 

' • ; ' vroulll rersult in an increase ''in post-test scores on ability and attitude. 

the teachsr activelv work© 



rrHoweveri- vintthis stud^ teacher actively worked with the class Vbef ore 

: ^f ; ^ and: after the- viewing, reinforcing the conteint, v;r; ; 

' Baseline Batteries . , . - , ' ^ 



y.' V;' ^ :• .V -'^^^ The -vyiewing groap-'scored significantly higher^than ..the inon-viewi^ 
{V;[/vi'f^^^^^^^ Parfo test, (See table 2) The^mean of the viewiilg 

• • • groi:ip:rr ( 31^ 85) was four points- hxgher than that: of thev noh-viewmg groupjcl 



Post- tests 



.*;, :: ; - The scores of /che two g^ were significantly, different on. two tests,^ - 

V i-s : • i : • vocabulary test and. the sentenced .comprehension -test . .{See table ;^2) • ,The A^t. 



scores- on the Jbther 



•^i-'^!^':";^^y^''"f6r^^ ' 



tests were . not significantly differejit/>after. adjusting \ 



A. 



ERIC 



S^iX--- ~T-r- — 



;ia.- 



,„ , ,„ 

aajus1:edCineans:',a!^ 



:f'c ' ' , ■.•/..Unadjusted: Means- 

iteS S^^ -^^^i^^^ -;:;:^'-:;:5 



;;i3;;W|[.;:',f;'65;;;0b::' 



Vocabulary, Test 

||:|ilMa^ 

!,CoristructionsiTest ■ 

St;i(Ma^i4)pi^§^^|f- 



; 8.13 



^6.90 



^-K/ ■■•!:. 



;| tl;;';UttiMei-Te^t'-3 ' ,'• :, 



[y,::;?v:-:j[Maj{=l^^ 









';;,':;;;;v.^i(:-..s.:L:::2i:^ 
















Ill 


















;:'';8i.79*:f;i/^dt;:;;" 


■au49::;' 






■:',i'.;.'..\!-':-<5|};' 


i'^::?''?4'53*-''a/70'-^;v' 




|;'||44;^.:; 
















■::^';i^.OO:v.:,l/70.'^-;;: 


:^45;'6^'':^^. 


|3|44.:Q7f:; 








;:8B.96.i 


;i;;8^.93,;:..:;,: 




. \ .■'..■..■■;:>lv-*'i.i;^ 














104^31 


■•■;i03Jl^;i 


5i:|i9i|ii|6i| 





■'pi'-"-.-;i"'-;'?'i-.'""-c'--.-':.,.<'- 





i:!;: ,; Study;3 . ■ ,. r. " ■ • ' 

i^-- V.-,:-. -r-v.rrrv ■•■ - - . ' : . , . - ■■ ■ ' ,■ -ir' i ^ 

0- '.: • . ^'^ ■ •■• • ; . ■>■>■•■■_ ■ • ' ■. . ' '■' • ■(?•'■ i'^ 

v '^j; ;:';';^ was 'tp determine ■thelef^ect .of a..teachdr ' ,;■}$ 

.■■';/on,th^ one group (TV) watched the- x^rbgram ' 

i '« without any- follow-up or . preparatory activity while . tlje- other group (Tvir. - ' ' ' 

^.■ir: .participated- in activities ■ in^ addition to viewing.' the program." • j; - V- ''i^iRv' • ■ ''- 'rr^ 

l^:.'^'-'^j'-:k Baselin^ -Test . ;• ' . - ; 

V\: 'TV : and Tejkcher .CTVT) ' igrovip /scored- 'sigrdfii^ the -Ici', 

gl^^^^ giP^P- ^4See-' table' ^ 

'''-^^ signi^icantJ^Y different on the' attitude* battery.^ "'At:' ."^ ' s'"^'']^^^^ 



the--. •/ -'^"^^ 



' ^ Vocabxaiary> Test ■ _ . ■. ' " ' 'V'tf'^N 

'-^ ■ 1 -^^ waa significantly diSfererit . from ,th^ ;T^^ v ■> L^' , 

V. ^ table./^^^^ The adjusted ^s^cores were 6151 for the^ T^ 

i i ^ .rThe^*ef forts of ; thfiL teacher pjDviously resiiltedtj in the increased vocabiilary ' j^: '^!^!^ 
. vy.; ■ level of ^;the .g;roT:5) who had'-the teacher adding her. input to the experienced - : ; v 



7 Sentence ^ Comprehension T;est. . ' ^-^ ^ ^ •••• 

- , . ■ T^^ scores for the two groups were not : significantly ctiffer.ent.^ 

(See ; table. 3) . The TV'iT group scored\at 50% (6.0). before adjus1:ment while the; 
;;-^:'TV group scored at: 30.5% (4. 74).. : — - , - 7-; : : _ i - . . ' 

* . /The efforts of the teacher, did- not app.ear to significantly affect the 
iresuits of the sentence compr-ehension test* . ■ - * ' , -V'- ' Vv :•• ; 



Cons tractions Test ^ ' . . * . 

The two groups were significantly different on the '^constructions test. 
(See ltable '3) . scores were adjusted, the TVT^group scorfed 9. 99 

/.while vthe jy. g^^ scores T.lSv Once again tha teacher preparatory and : • 
following activities appearedv to havG a positiv^^^ on the ability of the 

group to recognize correct constructions. ' a 

v.-. ^ : V..: ' ;^ ■ • ; •, . ■ ■■■» : ' • ^ ■■ ■■ ■■• ■ ■ ■•. • , .-.^ . ■•■:^:y/.- : 

Attitude Tests ' - ' *^ ^ 



V^-* On al^ -attitude testsV the groups did not differ significantly^ - 

(See taible :3).vvindicating that the intervention of a teacher alorig^^«?th the^ 
.televxsxoh. series did not significantly affect students' opinions. 



Table : : 3 



Ml ' O 



1'. : 



' ; Adjjisted and unadjusted maans/for . ' • 
TV only and TV. and Teacher Groupron' ali varia^^ 



■.A ■ (}•* , •! i r. 



Unadjusted, Means- 



!i.!Ad]usted' Means 



TV and' 



TV Only *' P. 'Ratio' ' df ' ■ ; TV Only „ ... , 
Teacher Teacher 



■ "-'''Ici-on .Parle .Test., 



" (Max:'40] 



Attitude tttlt'-:!';;^ 



!r;,i - 'Sentence Comprehension' 
r • .(Max42} >; •• 

^■■li'v'.. Vocabulary Test •. . 

[f"'?.;''''(Max;:16)"\ . ..V • .: 

jryr^-CoriBtructions Test 

:y v../, -''Attitude Test. 2 • , 

?i;:-':.''''(Max"s6ir,.', '." , 

/'Attitude -.Test 3 
::;5';\'-;iMax.=1551 |' - 



Test 



.25,31- 



60;24 



4.74 



6,06 



5,97 



31.85 3'7.43*''va/128'' ^ 



63^17 

r' ■ 

£iOO.' 
, 8.13' 

. r r ■ ■ 
■ i' ■ ■ 

il.77 



1 1 



.1,52 ■ '■•i/128 



14.98*' , 1/128; . 5.15 



5.-38' 



^ • . ' ' ; , \ -^'IV^^v 

.l/i28' - 6.51 ' ' .7.46 ':\ .^:86*-^.i/126;j^ 



70.39* ,"1/128 ■ 7.16' '' ''s'e.SS " 27;'26*; ".1/126, ;f? 



42.49--^.,. .45»29' 



'••■■■'-■4? 



1/128'- 43;19* '^^^4^.23:. ^• ; .62 . 1/126 "'^v.^;! 



■ 90.17 



. ... . ■ 'I 



8940— -SSS- 1/128 '89.79 ' 89.67 --'; 00 ; •1/126 7:;:' i^^^^^^^ 



•Aititud? Test! 4 ; 



"';.(Maxi80)\ '-' 



.104.18 . ■ ' i04.5f'' ■•• , i'04 l/i;8 ■i03.'44 ^^10S.65• \''^''lJ3^:,^l/l2.6;.'.'f^^^^^^^^^^ 



;;!!erjc.;:; ■••,„■■ , 




" I* ^ 1 ^ 



study 4 . 



} ^ ' * ' purpose of study four was to obtain student opinions of the various 

, production and content aspects of each of the ^^segments, ^The overSll findings 
^ ^seem to indicate-that jfor the most part, with the- exception of the Cheticainp 
segments/, the students were able to follow the action and understand what 
; was goxng on. Most of- 'the segments were considered quite eniovable ' . 

- , V l^ ' ! ' \ / . V " / ' ' 

r^' i '»by^ the students. ''They reacted positively to the teen-aged host and hostess 
/but said that some of the games on the game- segments were probably more 

^"^'^suited^-to 



I: 

because of the humour' and the quick 



^> action. " 



. .'V the recall« of segment answers indicated that most ^ 

stucients caught the^geneifal drif t ;of the action butrniissed important details; 



> IMPROVING EVALUATIONS: SOME "GUIDELINES ^ * . 

A varxety of experimental -designs and procedures are needed in order 

^/ ^ ^' ' . ^ 

to .conduct a formative eyaluajrion of a second language or indeed of any 
television project* I " - 

• vOne Of :;the .jnost ; iirportant factors "is the expert-: Evaluation in which ^ 



1 



people; who qualif ied :f or : the test review the program indeperifdent- of students 



to determjyse tne to questions such; as the suitability 'of the 



\ 



; yocabxilary, vthe accants/: .the "fcorrespdndence with the curriculum, the rp;lot 
^ level, etc. 



'^{^ ' -j, '^A^not^er prodedtore^i^ the use of a- feV students to determine 

\>\^. ^^^^^^ < " ' \ / ]^ ' • . \ ^ 

•vV 4!?7^^^^^^ program in comparison to the level* • 

.J of the^target ^audience ^ the attention of students in small groups and other 

^ -. such, factors. \s ' , "^'r' ' " ^ , ' \ 



1 ' 



field evaluation involves the actual in-class .test^g of the • 1 

-'""*,•, - , ... ./ . 

■.'S\ . program, with teacher input and witbaut, to determine the program's ab'lity -s'/-'' 
• ■ ■ '^V \ . ' , ■ ' 

: .J^' toNyachieve'^^its ^ intends objectives. This type of procecure was quite widely T i 

' . \ '^used in most evaluations'. ^ ' ' ; " ' ' - ' ' - 

:■' ^' ; \ ^ ' • . '■" • 

Ti- , Most ^evaluations' which included field testing use experimental group /~ ' 

control -srottp designs with or without some provision, for randomization or 
'?h ^~ Statistical /control of pre-treatment- differences. ' :^ ' ' ' ^ 



1 



' - N\ In ' evaluating second language materials, 'some or, all of the factors 
mentioned below may have to* be included. The factors . ar^" divided .into three ' " f 
^ categories: factors- which- can be examined by viewing %he'^progr^,^^=^ " - ^ 



^ factors . which need viewers interacting with the program.' * ^ 

' ^ " ' * ' * ' ^ . r - ,2^^ . ^'-^ 

Factors whxch can^ise examined independently ^ . , ^^'M 

. i/"^ .1. Vocabulary level bf'the program " . \ . ^ ' ^ 

^1"-- ' The vocabulary leirel of .^the. program must be within the' range of the^' ^ 
^ ' --^^l -students dn order that they dah benefit frqm itOie programs - Ther^a ar 

>ways, of determining '^ideal vocabulary levej'. One way would^ be to conpare the --T^ 
ti^^sV^H^. . 1^ wordS; used, in the progrcim with the- list of ^words used in. }the instruction r^^^^ 

ly.^.;^^ : -T being used "in the- curriculum. -'"^Another "means ^'miqht be to - ^ ' - ^ 

^ ' ■ - • ,^ • , ^- - ■ . . • _ 

; v pre-tfest. the students on the vocabulary- which is used in tiie program." The i^^^ -^^^^^^^ --^^^ 
pre-test can take the form which is used' in curriculum materials which-> • v ? 
' * ;;: ; i; usuerlly^ consists :^of a series of pictures in which the - student circles the correct*; 
; ^ :-L . alternative which is mentioned on an audio-tape. This' tyEfS of test was used: -^^^^^^^^^^ 
riA -^v-^' in both the "Vive <La Compagndbe" and the "Allons-Y" ^evalaia£ibns^. • A^variat^ --^ 
; : was used in the "Alloiis-Y" evaluation in which the student niimbered one of twent^^'' 

; : pictures with 1 the number ofi the phra^se or-word. ": However, - this test was :a biti^^^V'T 

' * • •■ . .i ■•• . :.. .; • • ■ --^ , ■ • • • . • . r s , . -■ : . . .■ . ♦ ■; , ' .« .. ,.• ■.. .... . ■.. ■;[.■■: ;ii 

• i : ' more diff iciilt -than 'the tra<iitional test in ^ which' the studerit circled "one *of ' '^^^^^^^^^^^ 
^ 'four alternatives. _ ; ^ - -'"tt^ '^^"l-t 



r^^vA. •2.':.;-; ^ of constructions used . 

i :i' ='\^^^^ effect comprehension of ■ students, 

'^■■■''•-^^ \ ' \\ 

The vfo??'^^^;'^^^^ in •the'progratm must- matph the* formats - . 

/ .which- the students" in their curriculum or"^hichvhave- been . V. 

. ^ reinforced in the . teaching, ..Once againy. the- types of constructions which^-tiie \ 
; ..student's imdefstand' can be t can be deduced from the ^program materials. 



V . 3,. - Accents . ■ ■' . ■ v. 

■•^-'/'-^ ^ ^ ■' . ■;- ■-r.^- .' \ ■ • • ■ 

The ^^ccent of the : actors , must m^tch those which the students are used - 
. to hearing. However, there is some; discussion on this fact with one caimp - 

! believing that -the student must get- used to hearing a variety. of ' accents - 

' . ' ' • • - ■■ 0 ■ . . - . ] ■ ■ ■ - . ■- ■ . .■• ' • " . ■" . '.■»•*.-■• • 

while the ioppo'Sit^ . view- holds that the student must^hear words"with .the ^ ' * . : ' • 
^. same accent . ' This is proven to be' an especially- vexing problem in Canada .. 

--r; y . ■ ■ . . , ^ ^ . ^ ^ ^ ^ / . 

. where the French spoken in the various regions differs ^ in terms of accent ■ 

■ ■ ■ - . ■ ^ . ■ ■ ■ ■ / ' ' 

and also differs from French spoken in France. Similarly, the types of , :. 

inflections placed ^'on words miist be similar to thosfe heard by the students..! ;. 

. One of the ways this task can be dpne is ta; ask e^ert personnel to./judge 

the aqcents^and inflections or to pre- test on a small niiinber of students, . : 

: ^^4 , ' /Image presented by. the -actors . . . ' ^ . - - . 

. In any materials which^'.have as a goal* thtf" improvement . of ' rela.tionships v ? 

:■" ♦ ■ :• •• .■■ ■ - . . ■ I ■ ••• .■ ■'■ , . ; , ' "■ •- ' ••: * . .. ■ •,■ . f--..--.'' i.-.--- •■ 

♦ between language groups, the "image presented by the actors in the second ^ ; 

/language program must be viery . carefully-monitorjed, •. Experts in ;the.:area ; '.y...:^^^ 
v. mii^' be asked *to evaluate the linage portrayed by the actors, ^Xf the actors 

* or actresses exhibit a , negative image to' the students, . this may. weil inhibit?'"^; 
iearning;, ' * / ■ . - ' . ' '\':.V-''Vi 



T. ::5v>i rcbtitri produbtioh' effects " ^ , 

;. : :The^^ production techniques should, be related to the' objectives 

: pf . should be; similar to the kinds 



whiph^the students are ^ccustWedr^Makeshif t-sets ar.e^ 



detected ve^^ students and cannot be used if the purpose'^is to * 

preseht^^^ In addition,/ tiie sets should not in them- 

;;selves dist^ objectives. ;• . » 



..v3: J 



^■^ V /^i o^ audio and visli^ meissage \ ' . : ; 

■ -fv^ presentation of the new vbcaipiilafy words can 'and should be accbirpanied; i> 

''r^-i::'-.}'^ 'their equivalent in a real referent, - When the French or Spanish word for. 

is^^ the/cup should be the most prominent object on the screen. Another: ■? 

.^ m^^ 1;o use a yisual cue, with perhaps one of the eictbrs pointing 

.i" " , . \.' ' ' - . ■ ■ - ■ ■ . ■' . . ■* . , . . ■ .•^ •. •*■-' 

■ . ;.. : •_ . . ■ ■ . . . . ■ . P . ' . >t ■*»•-•■..': y: 

^ to the cup, -to indicate precisely which object is being referred to in the 

.V one of the. programs, the, weak correspondence between . ' • i: 

audio ^^ ^^^ meant that students did not clearly understand which' 'i-^^^^^-.' 

. _ ^'p referring to. - . . ' 

■'. ■. , " . ■" . '.■ ■. ■. ' •' ' , , ■ ■ ■' , . ■ , • . ■ ' • ■ • ■ 

' : ■ , : 7. ! ^peed of the actors' speech ' * ■ 

■i-'- -'"l . ■ ■ "' ^ ;^ ' ^* * ..'^--h 

r . ; : is, correct for the stxidents. If .«v J 
■ V ; ' the piace is €00 quick the students will not understand the speech.. However, > • 
: if the;- speech is too slow, it seems unnatur^ and will, distract the students ■ 



8. . " .Level of content 



; ; ? V If . a story level df the "plot 'must match :the interest" 



';i::y;:i;:v^^ ■ "Allons-Y" a ' segmentv approach! is > used/:::'ffie;v:J|'0^ 




9» ; ;, C6mplekity of plot . 

The;plbt must be complex enough to =^hold the students' interest and 
"yet ; simple enough so that the students^^will understand it Teachers and . 
Other professionals can judge the . complexity of plot relative to a group, 
. however r^a~bettBr^-mekiis_mi^^ use small groups of . students an4 pilo^^^ 



. studies . to del termine^^^t^ plot to the intended a\idience.. ,v..v. - 

: * * . 10. Analysis of the program into infoarmatioh-^ units ^ ./ - >r:^l 

i ; ;Using ; a procedure^ outliijed by v Friedlander y program should: be > i^*; 

■ " -• • ■.' .■ ' ■'■ ■ , v./'.-*:..: 

broken into information units so that these \anits can be tested, dn coirp'rehension 

,tests or vocabulary tests later on.' Analysis of information units would 

!■'. . . . ' . • • . • . >• - 0 . ■ . ■ -K-^ ■ . .• . • ■■■■ . ■ 

— .. ■ •' • ■ • . ■ • ■ ■ ■ ■ * . . . " C ' ■ 

probably "take place -by ^teachers familiar ^with the program or by" researchers^ ; ■ 
who ^are conducting; the ^evaluation. - . , - v " 

Factors which must be examined, in' context . ' 

. ■ • • ' , ' 

1. Correspondence with the cvirriculum > ^ - ^- 

1 The- television program whxch is used should correspond iif some way fo ? 

. . 'V, • . . .• . . . , ^ . . ' ' . < ' 

the curriculiam which is. being used in 3the, schools. - If the vocabulary con- • 
structiojis- or structure"^ of the program is contrary to that of the instructional 
progrcomning used in the schools, a difficulty will emerge; Once again, this ; ■ ^ 
evaluation could be conducted Jiy expert researchers or by teachers \^o are' - • 
familiar, with both the program and the instructional materials being used' in'- - ^ 
the .schoolV - A <subsidiary .point, is that the program should Kelp fulfill v 
some 'Of .the curriculxim objectives in the curriculum. * It is important that/^^^ ^ 
the objectives of the program be somewhat the. same as. the objectives of the .- ^ 
overall ■curriculimi. ^ * v • l.'o 



fe^ ; program- -can "be :-.used^ r-.y.:/:^- ■ •;: ■ '-^^^i 

\ V;. In- mapy^^^ "the schedule of -the ? junior . high and ,high schools would ^ 

V the program to be used bn^ari of f-the-air T^asis^^/^ '''y ' '' ''-'i^^: 

; :> : V ' /^^ can^. be sirranged. ■ The ' abcessibility Vof the progifem : to^ ■ ^^^^^^^ 

; T^^^ must be: clearly ideritifiied. : In some of the^ 

jpgt ija^ ey alua t ions | i t wa s ^d i s c6 v er ed that because^ of a chain of 'circjmi^arices;fejg 

: : >;:V|mai^ prp<gram materials could sii^lyTiot^be-n^ 

i;^>,:iia to the inaccessibility of i^iijlmeht a^ problem 'of ' ; 

v<^'^:'J^r9tating:'*:shc ^- ■ . : ■' •V^'vr.:,-':^ 



/ 3«*- Preparatojry and follow-up woyk • v '' ^' 

• Through interaction' with teachers the ideal means of preparattory; and • ' : 
follow-up wc^k can and should be specified. This preparatory and follbw-t:^,, - S^^^ 
work' should help to achieve the program's objectives aind the objectives 6,f ; the 
overall curriculum. ^ One of the. ijnly ways" that this effect can be tiasted is_y:. . v?;-^^^ 
-to. contrast the use of teacher input with a compeirison group who do jiot have . 
teacher input such as was donia in * the "Alions-Y" : r^seeirch. z'^^, r ^'^ ^■^^ ' ' 



i '^;, ;; . 4^ V Teacher knowledge about using the program , v '-■^^'^rU ^'^^s 

Through direct contact with the teachers,, the- evaluation ; should - ^r'- '--':::^^ 

determine- whether teachers are able to use the programs or whether concrete O ; ;^ 
: . giiidelines^r should be provided. By having the 'teacher vieiw a number of programs' 
f and asking how the- ..programs would be iiised/ the evaluation, team, should be . v;^ 
able to determine whether concrete guidelines cire .needed to use the program,^ 



ERIC 



' r 5 , = .j. The' preparation of students ' • * .'. Vi^ 

V- ; :.,^The^ should try to determine whe€her concrete* preparation' "^^^ 

- of educational*^ television^materials is necessary for students. ^ '.^X 

:;V^ 'hv^i' ?^^ ^°''®d^cation to watch television but they may need- . - v -^^' 

' /[:^c5nieyHe^^ gaining the.m6st from ah educational teievisioh learning ' * / ^ .S? 



'-e2cperience. * . Z'*' 



' Factors which need students viewingrthe~ prograifl -—---.1:^ 
>■'• . " V;. 1 . , Attention ° . . - - 



l' Attention, interest or appeal of ^ tihej' program, is one of the f irst-^ • 
• variables to be . tested when students are involved. Attention can be measiared : i^-'!--^ 
by .various : means* ranging from, a simple count of*., eyes on. screen, to 'program -f^^" 
analyzer techniques, conjugate reinforcement "(Lindsley, O. R; 1962/ Ogston 1975) 'j 
. eye movement Studies (O^^^^ 1975) , distractor analysis (Palmer, 1974) ; or ' n^iSji 



similar techniques; Lewis (1973) reviewed various means of meateuririg atibentionWi; 
•to '■■audio- visual materials..- ■ .-.^ 



;.0 



: : ; .\The purpose of measuring this attention i s to ^ ensure that each segment" 

' - \ ' ^ ' * * ' *i - ' • ' . ■ ^ ^ '\ • ' ' . ' 

;:o:f: the program is interesting enough to be watched by the students. If the . ^ :^ 

Student J is not .watching the- program, all; future measurement will 'be in -''i'^^^:}-' 
^^^:•v■.■vjeopardy. i "; ' ^ , ' . : ^ ' . ' 

■ f ; :;;^A^ tested either in the grortp situatibn; or *inVa:. viewing -^^^-''-^ 

V- : - situatio one student or a ' small giroi^) . watching tie p 

.^^^.^Js inohitored by of ; the. techniques mentioned above to increase .the. yqca 

level of students, help them hear the spol^en language^ and' help them: ...V, ' 

ge i,* ^v^Tlie li^iM 




x:.'3v-^< Comprehension of the Program 

Of ten';:in the^^^^ t^^ ^at a level 



which can the understood, the complexity 



of the plot or* other .factors make. 



it hard for students to understand the exact course of action. ^As a result, 
.^the rcoirpreh^nsion of the program must . be There are various ;procedures^:!- 

::ito r assess -comprehension, : t^^^ simplest one being an interview with -the student 

'I I 

to determine what he or she remeicibered from the program. Other means which ^ / 
colilS be used include a question technique with mxiltiple alternatives, or a j 

^recall^-test'iin^ are simply : asked to^-write down what happened, . i '^^j^^ 



4, : Attitude to program segments - 
.:The enjoyment of the various segments or total program v is also one of 



the most important areas to measure. In! the case of -second langiaagevinstructiom^ 



. although- the program may fail to teach any vocabulary or construction or - i 

. even in fact may fail to be completely understood by students, they may . enjoy 

. ' ' 

the interaction with the program. The attitude to program;^segments also ^ ^ 

would- measure attitude to production varictblesr ^level of : plot, plot: complexity,r^^^ 

the actors and any other relevant information. ^ ^ ' v 



5. Attitude to production ^ ^ n 

The students who view our second language programs wiir have viewed^ a;- 



great; deaX of television by the time they interact w:^th one / of the programs 
that we produce. As a' result, \) it- is impor]:ant to <i^termine' whether /they r::^^^^ 



consider *the production level// to be similar to that of the commercials^ programs ^ot 



•"^ , that *they watch.' If the praductipn^levelyis seen to be inferior to the ^ ^ ' 




statements .or using Likert type scales. 



ERJC, 




vr:i::il -v-^>?: • 6 >;;io; P6strprbg2:am monitoring * 
' VtfTv;'- Many 'Of the effects of the second language program^ will -riot be . . : ^ ^ 

program is. viewed.^^ : 'H^ 
r:; ;;:^^^^ on after the students Jiave had .time to jdigest the program^^ -^Kp 

'.■ The purpose of p6st--progria .iactiyity 
■ 5::^: ; Mli^ to ;;detea:mine^the ; ability of tlie program ' to. produce chahges;.;iili: 

f So^^^^ thfB students. This usage of ^vocabulary Constructions :^'>^^ 

!j !v ; program/ tie repetition' of the types of activities prc^ced: - ^-^^^^^^ 



in the program wb'uiljd indica^te whether the^pi^ i ^. y- ■ ' 

successful in promoting long-term change. , Post-program mo nito'ring jactivity ''. 
.would have to take place ; through observational Jpriocedures br other sim 
rdevices. - . . •■■ ; r 



. 7. Attitude to French course. 
; / /^^^ expect that if one group of students is using a French television J 

the other, group is using standard^ ciirricular materials there '^ r 
- .- mightj^be a difference in attitiides - • Gardner and Snrythe' ; <.1975);; have suggested ^^^^^^^^ 
•; -.^yr-r^^-}:^ : ; an 1 attitude to French course , questionnaire which : coxild f probably be . used -to'y i: ^^jj:^^ 
r-::-. to the French course. . ' . — : ; 



Conclusions- 



- The formative evaluation* of bilingual ^television programs requires a'/- ' S'^':^;^ S 
ri v^^^^ approa.ch so that the various aspects of the; programs yi'.' 4^^^ 

- .'T^^^^^^^^ one approach, such -as field testing, can provide guidance^^lll 

• to producers : of television 




: I: vhave emsuiated from the research, "but have* not been fully tested. There is r ■ • ■•: 

• ^ If • . . ^ - . - 

v■--^^muc^^^ this field by researchers .in the field of . educational / 



;technology• .: ^ The. best research' designs and measurement techniques must be 
marshalled: t5o: advance the learning in this area'4 ■^■r^-- -r^'-.r..: 



✓ 
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